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Administrivia- Fall 2014 

August 26: 

There are a lot of you! 

4495A- about 62 undergrads 

4495GR - 40 grad students - mostly MS(CS) 

7495 - 40 mostly PhD students 

• Still working on the CS7495 model. We will not meet tonight - 
I’ll be working on your web site instead. Still need a room. 

• 4495: PSO can now be handed in. 

• 4495: PS1 will be out Thursday - due next Sunday 
(Sept 7) 11:55pm. 
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Linear outline (hah!) 

• Images are really functions I (x,y) where the vector can 
be any dimension but typical are 1, 3, and 4. (When 4?) 

Or thought of as a multi-dimensional signal as a function 
of spatial location. 

• Image processing is (mostly) computing new functions of 
image functions. Many involve linear operators. 

• Very useful linear operator is convolution /correlation - 
what most people call filtering - because the new value is 
determined by local values. 

• With convolution can do things like noise reduction, 
smoothing, and edge finding (last one is next time). 
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Source: S. Seitz 
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Images as functions 

• We can think of an image as a function, for I, 
from R 2 to R: 

f(x,y ) gives the intensity or value at position ( x, y ) 

Realistically, we expect the image only to be defined 

over a rectangle, with a finite range: 

f: [ a,b ] x [ c,d ] -> [0, 1.0] (why sometimes 255???) 

• A color image is just three functions “pasted” 

together. We can write this as a “vector-valued” 
function: r(x, y) _ 

f(x,y)= g(x,y ) 

b(*>y) 


Source: S. Seitz 
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» arnold(40:60,30:40) 

sns = 
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Digital images 


• In computer vision we typically operate on digital 
(discrete) images: 

• Sample the 2D space on a regular grid 

• Quantize each sample (round to “nearest integer”) 

• Image thus represented as a matrix of integer values. 
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Adapted from S. Seitz 
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Matlab - images are matrices 


>> im = imread (’ peppers . png ') ; % semicolon or many numbers 

>> imgreen = im( : , : , 2) ; 

>> imshow ( imgreen) 

>> line ( [ 1 512], [256 25 6] , ' color ' , ' r ' ) 

>> plot ( imgreen ( 25 6 ; 
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Noise in images 

• Noise as an example of images really being functions 

• Noise is just another function that is combined with the 
original function to get a new - guess what - function 

I \x, y) = I (x, y ) + fj (x, y) 


• In images noise looks, well, noisy. 
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Common types 


• Salt and pepper noise: 

random occurrences of 
black and white pixels 

• Impulse noise: random 
occurrences of white 
pixels 


of noise 



Original 



Sait and pepper noise 


• Gaussian noise: 

variations in intensity 
drawn from a Gaussian 
normal distribution 



Impulse noise 



Gaussian noise 


Source: S. Seitz 
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Gaussian noise 




/(® t y ) 


Id eal Imag e 

/Or, y) 



Noise process 



Gaussian i.i.d. { ‘'white' 1 ) noise: 

t)(x, y ) ■*- A'C^, t? ) 


» noise = randn( size (ini) ) . *sigma; 


» output = im + noise; 


Fig: M. Hebert 
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Effect of a on Gaussian noise 


Image shows 
the noise 
values 
themselves. 


noise = randn(size(im)).* sigma; 



Sigma = 2 


Sigma = 8 



Sigma = 32 


Sigma = 64 
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BE VERY CAREFUL!!! 


• In previous slides, I did not say (at least wasn’t supposed to 
say) what the range of the image was. A a of 1.0 would be tiny 
if the range is [0 255] but huge if [0.0 1.0]. 

• Matlab can do either and you need to be very careful. If in 
doubt convert to double. 

• Even more difficult can be displaying the image. Things like: 

• imshow(I, [LOW HIGH]) 

display the image from [low high] 


Don’t worry - you’ll get used to these hassles... see problem set 
PS0. 
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Linear Filtering/Convolution 
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Suppose want to remove the noise. . . 
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First attempt at a solution 

• Suggestions? 

• Let’s replace each pixel with an average of all the values 
in its neighborhood 

• Assumptions: 

• Expect pixels to be like their neighbors 

• Expect noise processes to be independent from pixel to pixel 


K. Grauman 
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First attempt at a solution 

• Let’s replace each pixel with an average of all the values 
in its neighborhood 

• Moving average in ID: 
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Source: S. Marschner 
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Weighted Moving Average 

• Can add weights to our moving average 

• Weights [1, 1, 1, 1,1] 15 



Source: S. Marschner 
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Weighted Moving Average 

• Non-uniform weights [1, 4, 6, 4, 1] / 16 


♦ ? 


I 


* ♦ • 


001464100 * 



Source: S. Marschner 
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Moving Average In 2D 
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Source: S. Seitz 
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Source: S. Seitz 
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Source: S. Seitz 
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Source: S. Seitz 
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Source: S. Seitz 
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Source: S. Seitz 
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Moving Average In 2D 

F [x, y] G [x . y 1 
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Source: S. Seitz 
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Correlation filtering 


Say the averaging window size is 2k+l x 2k+l: 


G[i, j ] 


1 


k 


k 


(2k +1)2 




F[i + u,j + v] 


U 
1 l 


k v 


k 


T 


Attribute uniform Loop over all pixels in 
weight to each neighborhood around image 
pixel pixel F[i,j] 


Now generalize to allow different weights depending on 
neighboring pixel’s relative position: 

k k 

G[i,j]= E E H[u,v]F[i + u,i + v] 

u= — kv=—k ' y 1 

Non-uniform weights 
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Correlation filtering 

k k 

G[i, j] = £ £ H[u,v]F[i + u,j + v] 

u= — k v= — k 

This is called cross-correlation, denoted G = H 0 F 

Filtering an image: replace each pixel with a linear 
combination of its neighbors. 

The filter “kernel” or “mask” H[u,v] is the prescription for the 
weights in the linear combination. 
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Averaging filter 

• What values belong in the kernel H for the moving 
average example? 

F\x , y\ 0 


G = H 0F 
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Smoothing by averaging 



depicts box filter: 

white = high value, black = low value 



original 



filtered 
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Squares aren’t smooth 


• Smoothing with an average 
actually doesn’t compare at all 

well with a defocussed lens 04 i 

0 . 35 -*, 

0 . 3 ^ 

0.£5v 

• Most obvious difference is that a 
single point of light viewed in a 
defocussed lens looks like a 
fuzzy blob; but the averaging 
process would give a little 
square. 

• More about “impulse” responses 
later... 
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Gaussian filter 


• What if we want nearest neighboring pixels to have the 



Source: S. Seitz 
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An Isotropic Gaussian 



The picture shows a 
smoothing kernel 
proportional to 




(which is a reasonable 
model of a circularly 
symmetric fuzzy blob) 


D. Forsyth 
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Smoothing with a Gaussian 




D. Forsyth 
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Smoothing with not a Gaussian 



D. Forsyth 
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Gaussian filters 

• What parameters matter here? 

• Size of kernel or mask 


• Note, Gaussian function has infinite support, but discrete 
filters use finite kernels 



10 X 10 
kernel 



o = 5 with 
30x30 
kernel 


K. Grauman 
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Gaussian filters 

• What parameters matter here? 

• Variance of Gaussian: determines extent of 
smoothing 



a = 2 with o = 5 with 

30 x 30 30 x 30 

kernel kernel 


K. Grauman 
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Matlab 

» hsize = 10; 

» sigma = 5; 

» h = fspecial( 'gaussian' , hsize. 


-« 




» outim = imfilter(im, h); 
» imshow(outim) ; 


sigma) ; 



outim 

K. Grauman 
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Smoothing with a Gaussian 


Parameter a is the “scale” / “width” / “spread” of the Gaussian 
kernel, and controls the amount of smoothing. 



for sigma=l:3:10 

h = fspecial( ' gaussian' , fsize, sigma); 
out = imfilter(im, h); 
imshow(out) ; 
pause; 
end 


K. Grauman 
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Keeping the two Gaussians straight. . . 

More Gaussian noise (like earlier) o -> 


ct=G.05 



CT=OJ 



o=0.2 



Wider Gaussian 
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And now some linear intuition... 

An operator H (or system) is linear if two properties hold 
(/ 1 and / 2 are some functions, a is a constant): 

• Additivity (things sum) (superposition): 

H(fl + f2) = H(fl) + H(f2) (looks like distributive law) 

• Multiplicative scaling (Homogeneity of degree 1) 

H(a • /l) = a-H(fl) 

Because it is sums and multiplies, the 
“filtering ” operation we were doing are linear. 
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An impulse function... 


• In the discrete world, and impulse is a very easy signal to 

understand: it’s just a value of 1 at a single location. 

▲ 





In the continuous world, an impulse is an idealized 
function that is very narrow and very tall so that it has a 


unit area. In the limit: 



► ◄ 
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An impulse response 

• If I have an unknown system and I “put in” an impulse, the 
response is called the impulse response. (Duh?) 



“Black Box” 




(or h(t)) 


• So if the black box is linear you can describe H by h(x). 
Why? 
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Filtering an impulse signal 

What is the result of filtering the impulse signal 
(image) Fwith the arbitrary kernel H? 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 



-V 

h 


II 



a 

b 

c 

d 

e 

f 

g 

h 

■ 

I 


H[u , v] 


F[x,y\ G[x,y] 


K. Grauman 
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Filtering an impulse signal 
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Filtering an impulse signal 
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Filtering an impulse signal 
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Filtering an impulse signal 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 



a 

b 

c 

d 

e 

f 

g 

h 

■ 

1 

m 

[u, 

v] 
























0 

f 



























F[x,y] 


G[x,y] 




CS 4495 Computer Vision - A. Bobick 


Linear Filtering/Convolution 


Filtering an impulse signal 
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Filtering an impulse signal 
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Filtering an impulse signal 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 



a 

b 

c 

d 

e 

f 

g 

h 

■ 

1 

m 

[u, 

v] 
























0 

f 

e 

d 

























F[x,y] 


G[x,y] 




CS 4495 Computer Vision - A. Bobick 


Linear Filtering/Convolution 


Filtering an impulse signal 
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Filtering an impulse signal 
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“Filtering” an impulse signal 



If you just “filter” meaning slide the kernel over the 
image you get a reversed response. 
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Convolution 


Convolution: 

• Flip where the filter is applied in both dimensions (bottom to top, 
right to left) 

• Then apply cross-correlation 


G[i,j] 



Centered at zero 



YH[u,v]F[i 


u,J 



u=—k v=— k 


G=H*F 


A 



Notation for 
convolution 
operator 


K. Grauman 
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One more thing... 

• Shift invariant: 

• Operator behaves the same everywhere, i.e. the value 
of the output depends on the pattern in the image 
neighborhood, not the position of the neighborhood. 
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Properties of convolution 

• Linear & shift invariant 

• Commutative: 

f * g = g * f 

• Associative 

(f * g) * h = f * (g * h) 

• Identity: 

unit impulse e = [..., 0, 0, 1, 0, 0, ...]. f * e = f 

• Differentiation: q 

dx 
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Convolution vs. correlation 

Convolution 

k k 

G[i,j]= E E H[u,v]F[i-u,j -v] 

u——k v— — k 

G = H * F 

Cross-correlation 

k k 

G[i,j]= E E H[u,v]F[i + u,j + v] 

u——k v——k 

G = H®F 

For a Gaussian or box filter, how will the outputs differ? 

If the input is an impulse signal, how will the outputs differ? 

K. Grauman 
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Computational Complexity 


• If an image is NxN and a kernel (filter) is WxW, how many 
multiplies do you need to compute a convolution? 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 



a 

b 

c 

d 

e 

f 

g 

h 

■ 

I 
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NxN 

• You need N*N*W*W = N 2 W 2 

• which can get big (ish) 
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Separability 

• Now we’re going to take advantage of the associative 
property of convolution. 

• In some cases, filter is separable, meaning you can get the 
square kernel H by convolving a single column vector by 
some row vector: 






1 

2 

1 

2 

4 

2 

1 

2 

1 
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Separability 



i 


2 

T 


r 



1 

2 

1 

2 

4 

2 

1 

2 

1 



G = H*F = (C*R)*F = C*(R*F) 

• So we do two convolutions but each is W*N*N. So this is 
useful if W is big enough such that 2 WN 2 « W 2 N 2 

• Used to be very important. Still, if W=31, save a factor of 
15. 
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Boundary issues 

• What is the size of the output? 

• Old MATLAB: filter2(g, f, shape) 


• shape = ‘full’: output size is sum of sizes of f and g 

• shape = ‘same’: output size is same as f 

• shape = ‘valid’: output size is difference of sizes of f and g 



Source: S. Lazebnik 
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Boundary issues 


What about near the edge? 

• the filter window falls off the edge of the image 

• need to extrapolate 

• methods: 

• clip filter (black) 



Source: S. Marschner 
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Boundary issues 


What about near the edge? 

• the filter window falls off the edge of the image 

• need to extrapolate 


methods: 

• clip filter (black) 

• wrap around 



Source: S. Marschner 
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Boundary issues 


What about near the edge? 

• the filter window falls off the edge of the image 

• need to extrapolate 

• methods: 

• clip filter (black) 

• wrap around 

• copy edge 



Source: S. Marschner 
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Boundary issues 


What about near the edge? 

• the filter window falls off the edge of the image 

• need to extrapolate 

• methods: 

• clip filter (black) 


wrap around 
copy edge 

reflect across edge 



Source: S. Marschner 
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Boundary issues 


What about near the edge? 

• the filter window falls off the edge of the image 

• need to extrapolate 


methods (new MATLAB) 

• clip filter (black): 

• wrap around: 

• copy edge: 

• reflect across edge: 


imfilter(f, g, 0) 
imfilter(f, g, ‘circular’) 
imfilter(f, g, ‘replicate’) 
imfilter(f, g, ‘symmetric’) 


Source: S. Marschner 
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Predict the filtered outputs 


nr 

0 

0 

0 

i 

0 

0 

0 

0 
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1 

1 



K. Grauman 
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Practice with linear filters 
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Original 



Source: D. Lowe 
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Practice with linear filters 
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Original 


Filtered 
(no change) 


Source: D. Lowe 
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Practice with linear filters 
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Original 



Source: D. Lowe 
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Practice with linear filters 



Original 
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1 
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0 




Shifted left 
by 1 pixel 
with 

correlation 


Source: D. Lowe 
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Practice with linear filters 
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Original 



Source: D. Lowe 




CS 4495 Computer Vision - A. Bobick 


Linear Filtering/Convolution 


Practice with linear filters 
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Original 


Blur (with a 
box filter) 


Source: D. Lowe 
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Practice with linear filters 
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Source: D. Lowe 
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Practice with linear filters 



Original 
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Sharpening filter 

- Accentuates differences 
with local average 


Source: D. Lowe 
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Filtering examples: sharpening 











T= 














before 


a iter 


K. Grauman 
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Effect of smoothing filters 



Additive Gaussian noise 


Salt and pepper noise 


K. Grauman 







CS 4495 Computer Vision - A. Bobick 


Median filter 
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Sort 


31 33 90 

I Replace 


Linear Filtering/Convolution 


• No new pixel values 
introduced 

• Removes spikes: good 
for impulse, salt & 
pepper noise 

• Linear? 


K. Grauman 
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Median filter 


Salt and 
pepper - 
noise 




◄ 




Median 

filtered 


Plots of a row of the 
image 


Source: M. Hebert 
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Median filter 

• Median filter is edge preserving 
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K. Grauman 
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To do: 


• Problem set 0 available; due 11:59pm Thurs Aug 29 th 

• Problem set 1 - Filtering, Edges, Hough - will be handed 
out Aug 28 th (Thurs) and is due Sun Sept 7, 11:59pm. 



